Fast and Space-optimal Low-rank Factorization in the Streaming Model With Application in Differential Privacy

In this paper, we consider the problem of computing a low-rank factorization of an m× n matrix in the general turnstile update model. We consider both the private and non-private setting. 1. In the non-private setting, we give a space-optimal algorithm that computes a low-rank factorization. Our algorithm maintains three sketches of the matrix instead of five as in Boutsidis et al. (STOC 2016). Our algorithm takes Õ(1) time to update the sketch and computes the factorization in time linear in the sparsity and the dimensions of the matrix. 2. In the private setting, we study low-rank factorization in the framework of differential privacy and under turnstile updates. We give two algorithms with respect to two levels of privacy. Both of our privacy levels are stronger than earlier studied privacy levels, namely that of Blocki et al. (FOCS 2012), Dwork et al. (STOC 2014), Hardt and Roth (STOC 2012, STOC 2013), and Hardt and Price (NIPS 2014). (a) In our first level of privacy, Priv1, we consider two matrices as neighboring if their difference has a form uv for some unit vectors u and v. Our private algorithm with respect to Priv1 matches the optimal space bound up to a logarithmic factor and is optimal in the terms of the additive error incurred. The algorithm is also efficient and takes time linear in the input sparsity of the matrix and quadratic in min {m,n}. Our bound quantitatively improve the result of Hardt and Roth (STOC 2012) by a factor of √ k log(1/δ) when m ≤ n, a scenario considered by Hardt and Roth (STOC 2012). (b) Our second level, Priv2, generalizes Priv1. In Priv2, we consider two matrices as neighboring if their difference has unit Frobenius norm. Our private algorithm with respect to Priv2 is computationally more efficient than our first algorithm – it usesO(log(m+n)) time to update and computes the factorization in time linear in the input sparsity and the dimensions of the matrix. This algorithm incurs optimal additive error and uses optimal space when n m. ∗Research supported by NSF award IIS-1447700. i ar X iv :1 60 4. 01 42 9v 3 [ cs .D S] 2 0 M ay 2 01 6